Establishing and maintaining perceptual coherence: unimodal and multimodal evidence

How does a listener find and follow the speech of a talker? Many classic and contemporary accounts of speech perception start with a coherent sensory sample of speech already established, as if the perceptual world consisted solely of speech, and as if no more than a single talker ever spoke at once. Long-established and recent characterizations alike have cast the fundamental problem of speech perception, of phonetic perceptual invariance despite sensory variability, in alignment with this assertion of coherence. The consequence of this theoretical practice has been to implicate the momentary acoustic cue and its sensory properties as the cause of phonetic perception. Our studies of perceptual organization of speech aimed to produce a direct characterization of the fundamental aspect of speech perception by which the listener resolves the acoustically heterogeneous signal elements into a speech stream. This work departed from the familiar concerns of phonetic perceptual analysis to ask, instead, about the primitive function by which a perceiver determines a sensory domain in which to apply analytic finesse. These studies reveal an aspect of speech perception, the perceptual organization of speech, that is fast, unlearned, intended, nonsymbolic and indifferent to the familiar sensory qualities of speech sounds. The theoretical yield of research about perceptual organization has two aspects. First, by examining the collection of momentary acoustic constituents into a time-varying pattern, it sets the terms by which to understand the perceptual resolution of acoustically diverse constituents into a single stream, or of the perceptual resolution of a speech stream amid other sounds, including the speech of other talkers. Second, by characterizing the establishment of a perceptual state in which phonetic analysis can occur, it sets the terms for describing a more realistic starting point for speech perception. A review of evidence from the perception laboratory establishes a common organizing principle at work in the perception of natural and synthetic speech, sinewave and noise-band vocoded speech, and chimerical derivatives of speech. Although it is clear that primitive auditory organizing principles deriving from the classic Gestalt set fail abjectly to accommodate the variety of speech signals, a claim of specialization in the auditory perceptual organization of speech is not warranted. Additional perspective on the problem is attained by considering multimodal instances of perceptual organization, which indicate that the detection of coherence across modalities shares formal properties with the detection of coherence within the auditory modality. The consequences of these findings for …